Jump to content

ISP Enabling and Usage

From RidgeRun Developer Wiki


Follow us on: YouTube Twitter LinkedIn Email Share this page

Share This Page

Qualcomm Dragonwing Technical Guide logo
Qualcomm Dragonwing Technical Guide

Qualcomm Dragonwing Technical Guide
Table of Contents [Sticky]



The IQ-9075 ISP path starts with a supported camera sensor. The qtiqmmfsrc GStreamer element exposes the ISP capabilities to adjust the image quality. This wiki provides verified example pipelines to test different ISP settings such as saturation, contrast and white balance. Performance results show that the simplest camera capture pipeline can keep up 4K@30fps and consumes around 13% CPU.


Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help .

Introduction

These pipelines were tested in the Ubuntu downloaded sources (Ubuntu 24.04.4). ISP use was verified within the kernel logs running the pipelines in one terminal and the following command in another terminal: 

sudo dmesg -w | grep -E "CAM-ISP|cam_ife|IFE|CSID|IPP|RDI|BPS|ICP"

When running the pipeline, the following output confirms the use of the ISP: 

[  305.799457] CAM_INFO: CAM-ISP: __cam_isp_ctx_acquire_dev_in_available: 7816: Acquire success: session_hdl 0xbe0200 num_rsrces -16843010 ctx 1 link: 0xffffffff
[  305.799626] CAM_INFO: CAM-ICP: cam_icp_v2_download_fw: 873: Loading Secure PIL : Y
[  305.830225] CAM_INFO: CAM-ICP: cam_icp_mgr_hw_open: 5018: [icp] FW download done successfully
[  305.830236] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2995: [icp]: FW_DBG:CICP_FW_E : HFI  :QC_IMAGE_VERSION_STRING=CICP.FW.7.8-00002,OEM_IMAGE_VERSION_STRING=CRM,BUILD_TIME: Apr 10 2025 23:00:32,CACHE_ENABLED at icphostinterface.c:880 QC_IMAGE_VERSION_STRING=CICP.FW.7.8-00002 OEM_IMAGE_VERSION_STRING=CRM
[  305.830242] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2995: [icp]: FW_DBG:CICP_FW_E : HFI  :ELF variant: T480:USE_CDM_1_1:TCM_ENABLED: , API version: 0x2490000 at icphostinterface.c:881 QC_IMAGE_VERSION_STRING=CICP.FW.7.8-00002 OEM_IMAGE_VERSION_STRING=CRM
[  305.840482] CAM_INFO: CAM-ISP: cam_ife_mgr_v_acquire: 18742: is FE 0 Offline 0 ctx id 0
[  305.840567] CAM_INFO: CAM-ISP: cam_ife_hw_mgr_print_acquire_info: 1730: 14:27:13.237 Acquired Single IFE[1] with [9 pix] [0 pd] [1 rdi] ports for ctx:1 per_port_enabled :0
[  305.840593] CAM_INFO: CAM-ISP: __cam_isp_ctx_acquire_hw_v2: 8263: Acquire success: session_hdl 0xbe0200s ctx_type 3 ctx 1 link 0xa50306 hw_mgr_ctx 1 is_shdr 0 is_shdr_master 0 offline_isp :0
[  305.841711] CAM_WARN: CAM-ISP: __cam_isp_ctx_process_evt: 9076: Get unexpect evt:2 in acquired state, ctx: 1 on link: 0xa50306
[  305.843255] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 2865: CAM_START_PHYDEV: 0, CSID:1, Type: DPHY, dev_cnt: 1, slot: 0, combo: 0, cphy+dphy: 0, skew_en: 0, sec_mode: 0, Datarate: 720000000, Settletime: 3200000000

ISP GStreamer Examples

The following table shows some of the qtiqmmfsrc element properties that can be tuned. See complete table in Qualcomm's Documentation.[1]

Property Description Values / Range
control-mode Top-level 3A control switch: auto-exposure, auto-white-balance, auto-focus. off, auto, use-scene-mode, off-keep-state
effect Effect applied to camera frames. off, mono, negative, solarize, sepia, posterize, whiteboard, blackboard, aqua
scene Camera optimizations for a scene. disabled, face-priority, action, portrait, landscape, night, night-portrait, theatre, beach, snow, sunset, steady-photo, fireworks, sports, party, candlelight, hdr
antibanding Adjusts exposure to avoid illumination banding. off, 50 Hz, 60 Hz, auto
sharpness Image sharpness strength. 0–6
contrast Image contrast strength. 1–10
saturation Image saturation strength. 0–10
iso-mode ISO exposure mode. auto, deblur, 100, 200, 400, 800, 1600, 3200, manual
manual-iso-value Manual ISO value used when iso-mode=manual. 100–3200
exposure-mode Camera exposure routine mode. off, auto
exposure-lock Locks current exposure values. true, false
exposure-metering Exposure metering mode. average, center-weighted, spot, custom
exposure-compensation Adjusts target brightness in steps. -12–12
manual-exposure-time Manual exposure time in nanoseconds, used when exposure-mode=off. 0–9223372036854775807
custom-exposure-table Custom exposure table structure. GstStructure
white-balance-mode White balance routine mode. off, manual-cc-temp, manual-rgb-gains, auto, shade, incandescent, fluorescent, warm-fluorescent, daylight, cloudy-daylight, twilight
white-balance-lock Locks current white balance values in non-manual modes. true, false
manual-wb-settings Manual white balance settings: color temperature or RGB gains. GstStructure
focus-mode Auto-focus mode. off, auto, macro, continuous, edof
noise-reduction Noise reduction filter mode. off, fast, hq
noise-reduction-tuning Noise reduction tuning structure. GstStructure
zoom Camera zoom rectangle in sensor active pixel array coordinates. <X, Y, WIDTH, HEIGHT>

Simple camera display

gst-launch-1.0 -e qtiqmmfsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! waylandsink

Adjusted camera

gst-launch-1.0 -e qtiqmmfsrc camera=0 white-balance-mode=6 focus-mode=1 iso-mode=0 exposure-mode=0 saturation=5 manual-exposure-time=43333333 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! waylandsink

Purpose:

  • Validate live video preview
  • Confirm camera bring-up
  • Verify NV12 output
  • Configure camera to white-balance-mode=6 (fluorescent), focus-mode=1 (auto), iso-mode=0 (auto), exposure-mode=0 (off), saturation=5 (default), manual-exposure-time=43333333

Camera with max saturation

gst-launch-1.0 -v -e qtiqmmfsrc camera=0 white-balance-mode=6 focus-mode=1 iso-mode=0 exposure-mode=0 manual-exposure-time=43333333 saturation=10 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! qtivtransform ! perf print-cpu-load=1 ! waylandsink

Purpose:

  • Change image saturation, by default is 5

Before

After

Camera with max contrast

gst-launch-1.0 -v -e qtiqmmfsrc camera=0 white-balance-mode=6 focus-mode=1 iso-mode=0 exposure-mode=0 saturation=5 manual-exposure-time=43333333 contrast=10 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! qtivtransform ! perf print-cpu-load=1 ! waylandsink

Purpose:

  • Change image contrast, by default is 5

Before

After

Camera with incandescent white balance mode

gst-launch-1.0 -v -e qtiqmmfsrc camera=0 focus-mode=1 iso-mode=0 exposure-mode=0 saturation=5 manual-exposure-time=43333333 white-balance-mode=5 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! qtivtransform ! perf print-cpu-load=1 ! waylandsink

Purpose:

  • Change white balance mode from 6 (fluorescent) to 5 (incandescent)

Before

After

FPS / Performance Display

gst-launch-1.0 -e qtiqmmfsrc camera=0 white-balance-mode=6 focus-mode=1 saturation=5 iso-mode=0 exposure-mode=0 manual-exposure-time=43333333 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! fpsdisplaysink video-sink=waylandsink text-overlay=true sync=false

Purpose:

  • Measure frame rate
  • Detect frame drops
  • Validate throughput

Fakesink Test

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! fakesink sync=false

Purpose:

  • Test video without display
  • Useful for SSH and performance measurements

Camera to H.264 Recording

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! v4l2h264enc capture-io-mode=4 output-io-mode=5 extra-controls="controls,video_bitrate=6000000,video_bitrate_mode=0;" ! h264parse ! mp4mux ! filesink location=camera-720p30.mp4

Purpose:

  • Validate Camera source → VPU encode path
  • Record video to MP4

Camera to H.265 Recording

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! v4l2h265enc capture-io-mode=4 output-io-mode=5 extra-controls="controls,video_bitrate=6000000,video_bitrate_mode=0;" ! h265parse ! mp4mux ! filesink location=camera-720p30-h265.mp4

Purpose:

  • Validate HEVC encoding path

Camera source with Qualcomm Transform

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! qtivtransform flip-horizontal=true ! waylandsink

Purpose:

  • Validate ISP → QTI processing
  • Check hardware transform path, it shows image flipped

Before

After


Camera with Split (Preview + Record)

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,width=1280,height=720 ! tee name=t t. ! queue ! waylandsink t. ! queue ! v4l2h264enc capture-io-mode=4 output-io-mode=5 extra-controls="controls,video_bitrate=6000000,video_bitrate_mode=0;" ! h264parse ! mp4mux ! filesink location=camera-preview-record.mp4

Purpose:

  • Simultaneous preview and recording
  • Validate bandwidth and buffering

Camera with Metadata Extraction

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,width=1280,height=720 ! qtimlmetaextractor ! fakesink sync=false

Purpose:

  • Validate metadata flow with GST_DEBUG=qtimlmetaextractor:5

Performance measurements

Camera simple measurements

The following table shows baseline camera performance using qtiqmmfsrc with NV12 format and no additional processing. GPU usage is not applicable in this configuration. 

gst-launch-1.0 -e qtiqmmfsrc camera=0 ! video/x-raw,format=NV12,width=$WIDTH,height=$HEIGHT,framerate=30/1 ! perf print-cpu-load=1 ! fakesink sync=false
Resolution %MEM RSS (KB) Mean FPS CPU (%)
1280x720 0.1 67652 29.948 12
1920x1080 0.1 67596 29.939 13
3840x2160 0.1 67584 29.947 13

Camera with qtivtransform and leaky queue

This table shows camera performance when adding a leaky queue and qtivtransform. This configuration introduces buffering and transformation but no GPU-based processing. This is the pipeline used for testing 

gst-launch-1.0 -e qtiqmmfsrc camera=0 ! video/x-raw,format=NV12,width=$WIDTH,height=$HEIGHT,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! qtivtransform ! perf print-cpu-load=1 ! fakesink sync=false
Resolution %MEM RSS (KB) Mean FPS CPU (%)
1280x720 0.2 103476 29.939 12
1920x1080 0.2 103492 29.923 13
3840x2160 0.2 103428 29.882 13

Glass-to-Glass Measurements

This section documents glass-to-glass latency measurements using different display sinks. In the case of kmssink, the display manager was stopped to fully own the display.

The camera used is a MIPI IMX577

Pipeline using waylandsink:

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! 'video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601' ! waylandsink

Pipeline using kmssink

Before running the tests with kmssink, stop the graphical interface and switch to multi-user mode: 

sudo systemctl isolate multi-user.target

Then run: 

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! 'video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601' ! kmssink

Results

Sink 4K latency 720p latency
kmssink 0.147339038 s 0.108750446 s
waylandsink 0.15126692 s 0.12861633 s

FAQ

How do I know the IQ-9075 ISP is active?
Check the dmesg for messages with the "CAM-ISP" string.
What to do if the image quality is poor even after tuning the qtiqmmfsrc parameters?
Driver bring-up and image tuning are separate tasks. Verify calibration, lens data, exposure, white balance, denoise, sharpening, distortion correction, and lighting conditions.

Related pages

Qualcomm Dragonwing Technical Guide logo
Qualcomm Dragonwing Technical Guide
Back to top ↑


  1. ↑ https://docs.qualcomm.com/doc/80-70018-50/topic/qtiqmmfsrc.html
Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=Qualcomm_Dragonwing_Technical_Guide/Multimedia_Support/Acceleration/ISP_Enabling_and_Usage&oldid=87013"
Cookies help us deliver our services. By using our services, you agree to our use of cookies.